1. Field
The present application relates to an image processing device and an image processing method for processing image data obtained by a two-dimensional image sensor mounted in a digital camera or the like at high speed.
2. Description of the Related Art
A two-dimensional image sensor is mounted in a digital camera, a color image scanner, or the like. Color filters of three colors corresponding to three primary colors of light are arranged in conformity with the arrangement of pixels of the image sensor. Electrical signals reflecting the light intensities of light components passing through these color filters are obtained at the respective pixels.
In a general two-dimensional image sensor, square pixels are arranged two-dimensionally, color filters corresponding to respective R, G, and B components are arranged according to the Bayer array. In this Bayer array, filters for passing therethrough light whose wavelength corresponds to the G component reflecting a brightness component are continuously arranged in an oblique direction. Furthermore, filters for passing light whose wavelengths correspond to the R component and the B component are alternately arranged on lines and columns.
In the image sensor having the color filters arranged in the Bayer array as described above, the pixel data of each pixel represents only the intensity of the component corresponding to the penetration wavelength of the corresponding color filter. Accordingly, the color at the position of each pixel of an image formed on the image sensor is obtained by interpolation with surrounding pixel data for each of the R, G, and B components.
As described above, the interpolation processing of obtaining image data having three color elements of R, G, and B at each of the respective pixels from a Bayer image obtained by the image sensor is called as deBayer processing.
When a Bayer image is obtained by a two-dimensional image sensor, the deBayer processing as described above is first executed to obtain a lineup of R, G, and B components for the respective pixels, and then various image processes such as a distortion correcting process, etc. are executed in many cases (see Patent Document 1: Japanese Patent No. 3549413).
In the image after the deBayer processing described above, all the pixels have the pixel data of the R, G, and B components. Therefore, in order to execute the distortion correcting process and a magnification converting process, it is required to execute a linear interpolation process on the pixel data of each of the R, G, and B components of one frame. This processing quantity is tremendous.
Furthermore, it is possible to execute the distortion correcting process and the magnification converting process on the image based on the Bayer array and then execute the deBayer processing to obtain an image for which distortion is corrected or image data on which the magnification converting process is executed.
However, when the distortion correcting process and also the magnification converting process are executed on the Bayer image itself, the following problems occur. The following problems are caused by the difference in feature between the arrangement of the R and B pixels on which the filters for passing light of the wavelengths corresponding to the R, B components are arranged and the arrangement of the G pixels on which the filters for passing light of the wavelength corresponding to the G component are arranged in the Bayer image.
One of the problems occurs in the case that the high pixel density of the G pixels in the Bayer image cannot be actively used, that is, when interpolation processing for correcting distortion is executed on the pixel arrangement data in which the G component in consideration of a coordinate system which has coordinate axes corresponding to the line and column directions of the pixel arrangement in the two-dimensional image sensor and whose unit corresponds to the arrangement pitch of the R pixels and the B pixels.
According to the above method, the interpolation calculation is performed in a grid formed by G pixels arranged at the same arrangement pitch as the R and B pixels as a unit. At this time, the pixel data of the G pixel located at the center of this grid is not reflected by the interpolation calculation for determining the pixel data at any position in this grid although the G pixel concerned is located at the center of the grid.
Furthermore, it may be considered that a grid formed by four G pixels arranged so as to surround an R pixel or B pixel is used and the interpolation processing of the pixel data of the G component at any position in the grid is executed in the Bayer image. According to this method, the pixel density of the G pixels can be actively used, however, the position calculation processing to correct distortion is complicated.